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Abstract 

Inspired by biological communication systems, molecular communication has been proposed as a viable scheme 
I to communicate between nano-sized devices separated by a very short distance. Here, molecules are released by 

the transmitter into the medium, which are then sensed by the receiver This paper develops a preliminary version 
I of such a communication system focusing on the release of either one or two molecules into a fluid medium with 

drift. We analyze the mutual information between transmitter and the receiver when information is encoded in the 
I time of release of the molecule. Simplifying assumptions are required in order to calculate the mutual information, 

and theoretical results are provided to show that these calculations are upper bounds on the true mutual information. 
I Furthermore, optimized degree distributions are provided, which suggest transmission strategies for a variety of 

drift velocities. 

; I. Introduction 

', Communications research has almost exclusively focused on systems based on electromagnetic prop- 
agation. However, at scales considered in nano-technology, it is not clear that these methods are viable. 
Inspired by the chemical-exchange communication performed by biological cells, this paper consid- 

. ers molecular communication in which information is transmitted by an exchange of molecules. 
Specifically we consider the propagation of individual molecules between closely spaced transmitters and 
receivers, both immersed in a fluid medium. The transmitter encodes a message in the pattern of release 
of the molecules into the medium; these molecules then propagate to the receiver where they are sensed. 

. The receiver then attempts to recover the message by observing the pattern of the received molecules. 
It is well known that microorganisms exchange information by molecular communication, with quorum 
sensing [2J as but one example, where bacteria exchange chemical messages to estimate the local popula- 
tion of their species. The biological literature on molecular communication is vast, but there has been much 
recent work concerning these systems as engineered forms of communication. Several recent papers have 
described the design and implementation of engineered molecular communication systems, using methods 

■ such as: exchanging arbitrary molecules using Brownian motion in free space [3]; exploiting gap junctions 

■ between cells to exchange calcium ions 01, JSl; and using microtubules and molecular motors to actively 
drive molecules to their destination [[61, |I3. A comprehensive overview of the molecular communication 
system is also given by jHl, [|9]| and the references therein. 

Given this engineering interest, it is useful to explore the theoretical capabilities of molecular commu- 
nication systems. To the authors' knowledge, the earliest effort towards information-theoretic analysis of 
these channels was given in IfTOl . which examined information flow in continuous diffusion. In IfTTTl . [fT2ll . 
physical models and achievable bounds on information rate were provided for diffusion-based systems. 

Manuscript received Jan. 12, 2010; revised Jun. 23, 2011; accepted Jul. 19, 2011. The associate editor coordinating the review of this 
paper and approving it for publication was Dr. M. Hughes. The material in this paper was presented in part at the International Conference 
of Computer Communications and Networks (ICCCN), San Fransisco, CA, 2009. 

S. Kadloor was with The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto. He is now with 
the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, 1308 West Main Street, Urbana, Illinois, USA 61801-2307. 
Email: kadloorl@uiuc.edu 

R. S. Adve is with The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 10 King's 
College Road, Toronto, Ontario, Canada M5S 3G4. Email: rsadve@comm.utoronto.ca 

A. W. Eckford is with the Department of Computer Science and Engineering, York University, 4700 Keele Street, Toronto, Ontario, Canada 
M3J 1P3. Email: aeckford@yorku.ca 



2 



Information rates were provided in [fT3l . [fT4ll for the case where the receiver chemically "reacts" with the 
molecules and form "complexes". In IfTSl . it was shown that the additive white Gaussian noise (AWGN) is 
appropriate for diffusion-based counting channels. Information-theoretic results have also been obtained 
for specific systems, such as propagation along microtubules fil6l . [|T7l and continuous diffusion [|T8l . 
All these studies indicate that useful information rates can be obtained, although much lower per unit 
time than in electrical communication; this is not surprising, since chemical processes are far slower than 
electrical processes. It is worth pointing out that these results build on theoretical work in Poisson and 
queue-timing channels |fT9ll , [|20ll . which is an active area of research in information theory. 

In any communication system, the potential rate of communication is determined by the characteristics 
of the channel. We consider molecular propagation in a fluid medium, governed by Brownian motion and, 
potentially, a mean drift velocity. Our model is therefore applicable to communications in, e.g., a blood 
vessel. This drift velocity is a key difference between our work and [fm . [[12], which considered a purely 
diffusion channel. Furthermore, we consider two cases - a single and two molecules being released. In 
this regard, it is worth emphasizing the preliminary and conceptual nature of this work. The long-term 
goal of this work is to understand the role of both timing and the number of molecules ('amplitude'). 
Thus, the contributions of this paper include: 

• Calculation and optimization, under some simplifying assumptions, of mutual information in Brow- 
nian motion with drift, where the transmitter uses pulse-position modulation; 

• Optimization of the degree distributions related to two transmit molecules; and 

• Demonstration (via theoretical results) that our simplified mutual information calculation is an upper 
bound on the true mutual information of any practical implementation of this system. 

Our optimized degree distributions reveal interesting features of these channels, suggesting transmission 
strategies for system designers. 

The paper is organized as follows. In Section [llj we describe the system under consideration, in which 
the propagation of the molecule is analyzed and the probability distribution function of the absorption 
time is derived. In Section [nil we characterize the maximum information transfer per molecule, for the 
case where information is encoded in the time of release of the molecule, and the case of two molecules. 
In Section [IVl numerical and theoretical results arising from these models (including optimized degree 
distributions) are presented. The paper wraps up with some conclusions and suggestions for future work. 

II. System Model 

A. Communication model 

The communication model we consider is shown in Figure [B The subsystems which make up the 
molecular communication system are: 

1) Transmitter. The transmitter is a source of identical molecules. It encodes a message in the time 
of dispersal of these molecules. We will assume that the transmitter can control precisely the time 
of dispersal of these molecules but does not influence the propagation of these molecules once 
dispersed. 

2) Propagation medium. The molecules propagate between transmitter and receiver in a fluid medium. 
The propagation is modeled as Brownian motion, and is characterized by two parameters: the drift 
velocity and the diffusion constant. These in turn depend on the physical properties of the fluid 
medium. The trajectories of different molecules are assumed to be independent of one another. 

3) Receiver. In this paper, the propagation of the molecule is assumed to be one dimensional. When it 
arrives at the receiver, the dispersed molecule is absorbed by the receiver and is removed from the 
medium. The receiver makes an accurate measurement of the time when it absorbs the molecule. 
It uses this information to determine the message sent by the transmitter. 

4) Transmission of information. The transmitter can encode information in either the time of dispersal 
of the molecules, or the number of molecules it disperses, or both. 
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Fig. 1. An abstract model of the molecular communication system. One or more molecules are released by the transmitter These molecules 
then travel through the fluid medium to the receiver, which absorbs them upon reception. If all the molecules are identical, then information 
is conveyed from transmitter to the receiver only through the times at which the molecules are released. 

The motion of the dispersed molecule is affected by Brownian motion; the diffusion process is therefore 
probabilistic and, in turn, the propagation time to the receiver is random. Even in the absence of any im- 
perfection in the implementation of a molecular communication system, this uncertainty in the propagation 
time limits the maximum information rate per molecule. In this paper, we study the maximum information 
per molecule that the transmitter can convey to the receiver, for a certain velocity and diffusion in the 
fluid medium. Before proceeding to do so, we need to characterize the propagation of the molecule in 
the medium. 

B. Diffusion via Brownian motion 

Consider the discrete-time, discrete-space propagation model in Figure |2(a)[ Let X(n) denote the 
position of the particle at time n. Let Px{x,n; Xo,no) denote the probability mass function (pmf) of 
the position of the particle at time n, given that it was dispersed in the fluid medium at position Xo at 
time rio- Assume that the fluid medium is static, and so the particle disperses in either of the directions 
with equal probability. If p is the probability that the particle moves from position x to position a; + / in 
one time unit, and q is the probability that it moves from position x to x — I, then this situation is the 
case when p = q = 0.5. It is easy to see that Px{x., n; Xo, rio) obeys the equation 



which states that if a particle at time + 1 is at position x, then at time n, it should have been at position 
X — I or X + 1, where / is the distance between two slices of space. This formulation of Brownian motion 
is analogous to a Wiener process, where distinct increments of the motion are independent from each 
other. 

Equation ([T]) can be re-written as 

Px(x, n + 1; Xo, rio) - Px{x, n; Xo, rio) 

1 1 
= -^{Pxix - I, n; Xo, rio) - Px{x, n; Xo, rio)) + -j{Px{x + /, n; Xo, rio) - Px{x, n; Xo, rio)) 

P fl f Px{x - l,n;Xo,no) - Px{x,n;Xo,no)\\ 



Px{x,n + l;Xo,no) = -Px{x - l,n; Xo,n^ + -Pxix + l,n;Xo,n^, 



(1) 




(2) 



4 






Time 

(b) Sample paths of six particles in the same fluid medium, three released 
at t = 0, three a.1 t = 400. 

Fig. 2. If the size of the receiver is several orders greater than the size of the molecule, and if the velocity of the fluid in the 'y axis' is 
negligible compared to the velocity of the fluid along 'x axis' (Refer Fig. [TJ, then, one can ignore the y-coordinate of the position of the 
molecule and consider only the x-coordinate. The position of the molecule along the x axis is modeled as a Markov chain, specifically, as 
a one dimensional random walk. The bias of the walk (the values of p and q) depend on the velocity of the fluid medium along the x-axis. 



When n ^ 1 and x ^ I, the difference equation becomes a continuous time differential equation, yielding 
a probability distribution function (pdf) for the position of the particle, given by, 

d f d'^ 

-7—Px [x, n; Xo, no) = —tt^Px {x,n;Xo,no). (3) 
on 2 ox^ 

Now, considering a continuous time Brownian motion X(t), the probability density function of the position 

of the particle can be modeled by the diffusion equation 

d 

— Px(a;, t; Xo, to) = D—Px{x, t; Xo, to), (4) 

where D = P /2 is the diffusion constant, whose value is dependent on the viscosity of the fluid medium. 
Note that the above equation characterizes only the 'x-coordinate' of the position of the molecule. Solutions 
to this equation are well known. 
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Equation dH) characterizes the motion X{t) of the particle in a macroscopically static medium. The 
more general and useful case is that of a fluid medium is in motion with a mean drift velocity v. Consider 
a frame of reference which is moving with the same velocity. In this frame, the fluid medium is static 
and hence the diffusion of the particle should obey Equation (HI). Let 

X = X + vt, t = t 

be the new coordinate system, and without loss of generality, assume to = 0. Let 

Pxix,t;Xo,0) = P^/(x',t';Xo,0), 

then 



'q^ X' V"'' ' ^ i ^O' ^) ~ X' ' ^ ' -^o, 0). 

In the static frame of reference, the differential equation can be written as 



dill ^ r>' I ' ' n^ 

— P^/(a; ,t ;xo,0)^ + —P^i[x ,t ■,Xo,0)-^ 

^ d f f dx d dt d\ I , I ^, , 

^7^7 7^77^ + 7^77^ ^0,0) , (5) 



which simplifies to 



dx \ \ dx dx dx dt J ^' 
^Pxix, t- Xo, 0) = (d^ + ] Pxix, t- Xo, 0). (6) 



Ot \ OT^ OX 



Assume that there is no absorbing boundary (receiver) and that the fluid medium extends from — oo 
to +00. The probability density function of the location of the particle can be obtained by solving the 
differential Equation Q with boundary conditions Pxix, 0; Xo, 0) = 5{x — Xo) and Px{±oo, t; Xo, 0) = 0. 
The solution to © is given by [1211 

1 / (x-vtf\ 

Equation (|7]) states that, for every t, the probability density function (pdf) is a Gaussian centered at vt 
with variance 2Dt. As expected, the expected location of the particle drifts along the direction of flow 
of the fluid medium with velocity vt. Figure [3] plots P{x,t) for the case when v = 3 and D = 0.3. 
Furthermore, for any transmitter point and transmit time to^ we have that 

As expected, Brownian motion X{t) satisfying ©-([S]) is a Wiener process with drift. 

Now, consider the case when there is an absorbing surface (receiver) at x = 0. The particle is absorbed 
and is removed from the system when it hits the absorbing surface. For such a system, to solve for 
Px{x, t; — C, 0)), we need to solve the differential equation in ^ with the following boundary conditions. 

• For X < 0, Px{x, 0; 0) = 5{x+C)- The probability density function has a physical interpretation 
only for x < 0. In this region, we require it to be a delta function at t = centered at x = —(^. 

. Px(-oo,t;-C,0) = 0, Vt. 

• Px(0,t; — C;0) = 0, Vt. Condition imposed by the absorbing surface. 

The solution to the differential equation can be computed using the method of images, it is given by: 

^"<"' °' = TSB?'"" \ — - TSsf'^P [ — w^) [d) 
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Simulation parameters: 

Velocity, v=3 

Diffusion constant, D=0.3 



-T=0.01 
-T=0.25 
T=0.5 
■T=0.75 
■T=1 




-4-2 2 

Position of the molecule, x 



Fig. 3. The pdf of the position of the molecule P[x,t) for different values of t, when it is released at time t = Q a\. position a:: = 0. 
Because of positive drift velocity, the mean of the pdf travels in the positive direction, and because of the diffusion, the variance of the pdf 
grows with time. 



C. Distribution of Absorption Time 

Recall from Section |ll] that the receiver senses the particles only when they arrive, at which time they 
are absorbed and removed from the system. Thus, for the purposes of this paper, the most important 
feature of the Brownian motion X{t) expressed in ©-([H]) is the^r^f passage time at the destination. For 
a Brownian motion X(t), and an absorbing boundary located at position C, the first passage time t{Q at 
the barrier is defined as 

r(C) = m^in{X(t) : X{t) = Q. (10) 

In Figure |2(b)[ the simulated trajectories of six particles, modeled as a random walk, through a medium 
are plotted. The particles were all released at x = 0, three at time and three at time 400, into a fluid 
medium that had a positive drift velocity. The receiver is located at x = 50. Notice the large variation 
in the absorption times. Among the particles released at t = 0, one gets absorbed at t ^ 360, other at 
t ^ 500, and another does not get absorbed even hy t = 1000. Furthermore, this plot shows how particles 
can get absorbed in an order different from the order in which they were released. It is therefore important 
to understand the variation in the propagation times of the particle. 

The derivation of the first passage time for our case is given in [|22|. Here, we repeat briefly the steps 
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involved. At a given time t, the probability that the particle has not yet been absorbed is given by 

fO 



F{t) = I Px{x,t;-C,0)dx 

{x + C~ Vtf 




dx 

(x - c - vty 

4Dt 



dx 




F(t) is the probability that the particle has not been absorbed until time t. The probability that the particle 
has been absorbed before t is given by F{t) = 1 — F{t). Hence, the probability density function of the 
absorption time is fit) = F' (t) = -F'(t). 

fit) = - 





\V2Di 2V2Dt3 
-{vt + C?\ f -V ^ jvt + C) 
\V2Di 2V2Dt^ 

^^A^""^ V 4Dt ' ^^^^ 

To summarize, (fTTI) gives the probability density function of the absorption time of a particle released in 
a fluid medium with diffusion constant D, at a distance C, from the receiver, when the fluid has a constant 
velocity v. Note that this equation is valid only for positive drift velocities, i.e., when the receiver is 
downstream from the transmitter. Since our communication is based largely on the time of transmission 
(and reception), this pdf characterizes the uncertainty in the channel, and plays a role similar to that of the 
noise distribution in an additive noise channel. Some example plots of this function are given in Figure IH 



III. Mutual Information 

The transmitter encodes the message in the time of release of molecules and possibly the number 
of molecules. Based on the number and the time of absorption of the molecules, the receiver decodes 
the transmitted information. This section develops the mutual information between the transmitter and 
receiver for two cases: with a single transmitted molecule and two molecules whose release times can be 
chosen independently. For a given information transmission strategy at the transmitter (called the input 
distribution in the information theoretic literature), the mutual information is also the maximum rate at 
which information may be conveyed using that strategy. (Mutual information is related to but distinct 
from the capacity, which is the maximum mutual information over all possible input distributions.) 



A. Overview 

In a traditional wireline communication system, receiver noise causes uncertainty in the reception, 
limiting the rate at which information can be conveyed. However, as discussed before, the uncertainty 
in the propagation time is a major bottleneck to the information transfer in molecular communication. 
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Fig. 4. The time at whiich the molecule gets absorbed by the receiver, given that it was released at time 0, is a random variable. This is 
a result of the diffusion of the fluid medium. Here, we plot the probability distribution function of the absorption time for different sets of 
velocity and diffusion. For this plot, the distance between the transmitter and the receiver is set at 1 unit. 

This uncertainty in the propagation time also means that the order in which molecules are received at the 
receiver need not be the order in which they were transmitted. This will result in "inter-block interference". 
This is a serious impairment in the low velocity regime, where the pdf of the absorption time decays very 
slowly, making inter-block interference more likely. 

In this paper, we ignore inter-block interference and assume that the clocks are synchronized. Developing 
techniques for both issues are significant works in themselves and outside the scope of this paper. So our 
results are most relevant to system in a fluid with some significant drift; further, as we show in Section 
rvl our results can be used to obtain upper bounds on both mutual information and capacity for any drift 
velocity. 

The channel here falls under a class of timing channels, channels where the mode of communication 
is through the timings of various events. The capacity of such channels are usually more difficult to 
characterize. A celebrated result in this field is the computation of the capacity of a single server queuing 
system [20J. The molecular communication channel can be modeled as a -/G/oo queuing system, i.e., an 
infinite server queuing system where the service time of a server is a random variable with distribution 
same as the pdf of the absorption time. To our knowledge, the exact capacity of such a channel has not 
been computed to date. 

B. Single molecule: Pulse position modulation 

We first analyze the case of the transmitter releasing just a single molecule. In such a scenario, it can 
encode information only in the time of release of the molecule. The transmitter releases the molecule (or 
not at all) in the beginning of one of N time slots, each of unit duration (i.e., = 1 in arbitrary units); 
this action on the part of the transmitter is called a channel use. This molecule then propagates through 
the medium and is absorbed by the receiver in a later time slot. The receiver then guesses the time slot 
in which the molecule was released. This is a form of pulse-position modulation (PPM). 

Given that it has (A^ + 1) choices, the transmitter can encode a maximum of log2(A^ + 1) bits of 
information per channel use, though in practice much less due to the uncertain arrival times of the 
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molecules. For instance, suppose the velocity of the fluid medium is high enough so that the particle gets 
absorbed by the receiver in M ~ time slots with very high probability. In this case, one transmission 
strategy would be to emit a molecule in one of N/M time slots (each separated by M slots), since inter- 
block interference would thus occur with very low probability, and the transmitted information would arrive 
without distortion. For such an ideal system, we can transmit information at a rate of loggA^/M + 1 bits 
per channel use. However, more practical and interesting is the less than ideal case with lower velocities. 

In this paper we neglect inter-block interference, i.e., we assume that the receiver waits for enough time 
slots A/, to ensure that the molecule propagates to the receiver with high probability. Here M is chosen 
such that this probability is 0.999. Further, we assume that the receiver sampling rate is T,. = T^/S. This 
provides both a digital input/output system while maintaining fairly high accuracy of the received time. 
Both these parameters could be changed as required. 



C. Mutual information as an optimization problem 

Having dealt with preliminaries, we now derive the maximum possible mutual information, here as 
an optimization problem. Define a random variable X to denote the time slot in which the transmitter 
releases the molecule. Assume that the transmitter releases the particle at the beginning of the i*^ slot 
(1 < i < A^) with probability Pi. With probability p^ = \ — Yl!i=iPi^ transmitter does not release 
the particle. Let Y denote the time slot in which the receiver absorbs the molecule. For the time being, 
we allow Y to range between 1 and oo, we will see shortly that this is not required. Also, let F = 
denote the event that the molecule is never received. Since in our idealized case, the receiver waits for a 
sufficiently long time, the event of F = is the same event that the molecule is not transmitted. Assume 
that the duration of the time slot is T,.. 

Let F(t) denote the probability that the particle gets absorbed before time t given that it was released 
at time 0, i.e., F{t) is the integral of the pdf in (fTTI) . Denote by aj the probability that the particle arrives 
in the j^^' time slot, given that it was released at time 0, which is equal to F{jTr) — F({j — l)Tr) ; 
«j = 0, j < 0. Let H(X) denote the entropy of random variable X and let entr(x) represent the binary 
entropy function, where 

entT{x) = \ (12) 
10 X = 0. 

We now proceed to calculate the mutual information between the random variables X and Y. 

N 

H{Y\X) = H{Y\X = 0)po + Y,H{Y\X = i)pi 

1=1 

N 00 N 00 

= X po + ^pi entr {P{Y = j\X = i)) = J^Pi X] ^^^^ 

1=1 j=i+l 1=1 j=i+l 

00 

= (1 -po)X^entr(afc) , (13) 

k=l 

H{Y) = entr(P(F = 0)) + ^ entr(P(r = j)) 

00 / N 

= entr(po) + entr j P{Y = j\X = i)pi 
j=i \i=i 

00 / N \ 

= entr(po) + ^ entr I ^ {uj^i) Pi j (14) 
0=1 \i=i ) 
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I{X-Y) 




entr(po) + ^^entr j ^^pjO; 

j=i \i=i 



) 



(1 -po) X^entr (a^) 



k=l 



oo 



(15) 



As seen in Figure HI the sequence is a an eventually decreasing sequence. The rate of decay depends 
on the values of the drift velocity v and the diffusion coefficient D. The summations in (fTSl ) can, therefore, 
be terminated for some large enough M. 

The expression for mutual information is a non-negative weighted sum of concave functions plus a 
constant. Hence, the mutual information is a concave function of the input distribution {pi,i = 1, . . . , A^}. 
Finding the degree distributions, the values for piS which maximize the entropy, is therefore a concave 
optimization problem. Standard convex optimization techniques can therefore be used to solve for the 
input probability distribution which maximizes the mutual information efficiently, in particular, the Blahut- 
Arimoto algorithm [23l, [l24l . 

As a special case, suppose that we were to convey information only in the time of release of the 
molecule, i.e., we require the molecule to be transmitted. The derivation of mutual information is very 
similar to the derivation above. Mutual information can then be expressed as 



and, again, the optimal degree distribution can be obtained through concave optimization. 
D. Two molecules 

In the work so far we have considered only the propagation of a single molecule and the focus was on 
PPM-based communication. We now take a step toward involving amplitude wherein the transmitter can 
release two identical molecules. The analysis is simplified by assuming that the propagation paths of these 
two molecules are independent. The transmitter releases each of these molecules in one of the N time 
slots or chooses not to release it. Based on the arrival times of these molecules at the receiver, the receiver 
estimates their release times. However, because of the nature of the diffusion medium, different molecules 
can take different times to propagate to the receiver. Hence, the molecules can be absorbed in a different 
order than in which they were released: a key difference between this channel and traditional additive 
noise channels. As a result, the amount of information that can be conveyed through the medium with 
two indistinguishable molecules, as we will shortly see, is less than twice the amount of the information 
that can be conveyed using a single molecule. 

To obtain the maximum mutual information, let Xi G {1, 2, . . . , N} be the time slot in which the first 
particle is released, X2 E {Xi, Xi + 1, . . . , N} be the time slot in which the second particle is released. 
Let Yi, and Y2 be the time slots in which the first and second particles are received. For notational 
convenience, if a particle is not released, we denote it by a release in slot 0. Likewise, if a particle is not 
received at the receiver, we denote it by a reception in time slot 0. 

The probability mass function of the reception times {PiYi, Y2)), and the conditional probability mass 
function of the reception times given the transmission times (^(^1, 5^2|-^i5 -^2)) can be expressed in terms 
of the conditional probability mass function of the reception time of one molecule, given its transmission 
time {P(Yi = yi\Xi = xi) = ay^-^^). Let p^^^^ represent P{Xi = xi,X2 = X2). 




(16) 
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P{Yi = yi,Y2 = 0\Xi = xi,X2 = 0) = ay^-^^, xi,|/i>0 
P{Yi = y,Y2 = y\Xi= xi,X2 = X2) = ay.^^ay.^^, Xi>X2,y>0 
P{Y,=y,,Y2 = y2\X,=x,,X2 = X2) = 

oiy^-x2 + ajyi-x2 "2/2-2:1, 2:1 > a;2,?/i 7^ 2/2 > 

P(Fi = 0,F2 = 0) = Poo 



xi=l 
N N 

P{Yi=y,Y2 = y) = ^ ^ p^.^^lay-^^ay-^,) 

Xl = l X2=Xl 

N N 

PiYi = 2/1, >2 = 1/2 7^ 2/1) = X] {^yi-xiay2-x2 + otyi-x2ay2-x^) 



Xl = l X2=X\ 

The term ay^-x2Ciy2-xi in the above equations accounts for the event that the molecule released later 
gets absorbed before the molecule which is released earlier. The mutual information between the variables 
(Xi,X2) and (Yi,Y2) can now be written in terms of these probability mass functions. Note that in the 
above derivation, we have assumed that ak for A; < is defined as zero. 

Using these equations, we can frame the mutual information maximization as another optimization 
problem. The optimization is to be done over the upper triangular N x N matrix Pxi,X2ixi^ ^2), where 
each entry in the matrix is a positive number and all the entries sum to one. The mutual information is 
a concave function of the optimization variables {pxiX2 : Xi G {1, 2, . . . , A^}, X2 G {xi,Xi + 1, . . . , N}}. 
The exact expression is tedious to write, and is omitted here. 



IV. Results 

The well known Blahut-Arimoto algorithm [|23l . [|24l| is used to compute, numerically, the input 
distribution that maximizes the mutual information in each of the different scenarios. The distance from 
the sender to the receiver, ( is set to one unit in all the results presented here. 



A. Release of a single molecule 

When one molecule is to be released, information can be conveyed in whether it is released or not, 
and if released, the slot number in which it is released. 

Case when the molecule can be released in one of the N slots, or not at all: In Figure [51 we 
plot the mutual information as a function of velocity, for two different sets of diffusion coefficients, 
0.05, representing the low diffusion scenario, and a high diffusion constant 0.2. We have two sets of 
plots in the figure, one for the case where we have two slots in which we can release the molecule, or 
choose not to release it, and another, where we have four time slots. Also, we give the input distribution 
{Po^PiiP2^Pz^Pi) at which the mutual information is maximized at the two extreme values of velocity. 

From the figure, it is evident that the mutual information increases with an increase in velocity and 
saturates to a maximum of log2(A^ + 1) bits. This trend is as expected. At high velocities, the optimal, 
information maximizing, distribution is uniform. This is because the receiver can detect, without error, 
the slot in which the transmitter disperses the molecule. Also, because the receiver waits for a sufficiently 
long time, we can detect, without error, if a molecule was transmitted or not. Therefore, a lower limit 
on the mutual information is one bit. At lower velocities, timing information is completely lost and the 
mutual information is marginally greater than one bit. 
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2.5 




Velocity 



Fig. 5. Variation of mutual information (whicli measures in bits, the information tliat can be conveyed from tlie transmitter to tlie receiver) 
with velocity. There are 2 sets of curves corresponding to the number of slots in which the molecule is released, N = 2 and = 4. For 
iV = 4, we also list the p.m.f. of the release times which maximizes the mutual information. 

The diffusion constant is a measure of the uncertainty in the propagation time. Hence, we would expect 
the mutual information to be lower when the diffusion constant is high. This is indeed the case at high 
velocities. However, it is surprising that a higher diffusion constant results in higher mutual information 
at low velocities (Also refer Figure (6]). This is because, at low velocities, it is the diffusion in the medium 
which aids the propagation of the molecule from the transmitter towards the receiver. This is illustrated in 
the pdf of the absorption time, shown in Figure IH Compared to the case when the diffusion in the medium 
is low, the probability distribution function is more "concentrated" (lower uncertainty) when the diffusion 
in the medium is higher. Unfortunately, there does not seem to be a single parameter that characterizes 
the resulting interplay between velocity and diffusion. 

Case when we do not permit the transmitter to not transmit the molecule: The information, in 
this scenario, is conveyed only in the time of release of the molecule. We find the input distribution 
which maximizes (fT6l ). The mutual information in this case is plotted in Figure |7l The maximum mutual 
information is now log2(A^) bits, which is achieved at high velocities. However, it is in the low velocity 
regime where the mutual information is significantly lower than the case where the transmitter is allowed 
to not transmit the molecule. Figure [8] compares the two scenarios. 

From the results, we see that the velocity-diffusion region can be roughly classified into three regimes: 

• A diffusion dominated region, where mutual information is relatively insensitive to the velocity; this 
corresponds to f < 10^^ in Figure [51 

• A high-velocity region where the mutual information is insensitive to the diffusion constant; this 
corresponds to t> > 3 in Figure [51 

• An intermediate regime, where the mutual information is highly sensitive to the velocity and diffusion 
constant of the medium, 10^^<t'<3in Figure [51 

In the low velocity regime, we see no significant improvement in the mutual information when we 
increase the number of time slots in which we can release the molecules. As expected, very little 
information can be conveyed in the time of release of the molecule when there is high uncertainty in the 
propagation time. Hence, we need to explore alternative ways of encoding message in this regime. 
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Fig. 6. A grid plot denoting the mutual information for a range of different velocities and diffusion constants, for the case when A'^ = 4. 
Observe that at lower velocities, more information can be transferred in a medium with higher diffusion constant. 




Fig. 7. Variation of mutual information with velocity when the transmitter must disperse the molecule (po ~ 0). The scenario is similar to 
the one used in plotting Figure [3] with the difference being that the transmitter is not permitted not to transmit a molecule. 
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Fig. 8. A comparison of the information bits conveyed in the scenarios when the transmitter must (po — 0) or may not release the molecule. 
Plots from Figures |5] and |7] are compared here. 



B. Release of multiple molecules 

Here, we present the results of the scenario in which the transmitter is allowed to transmit at most 2 
molecules. The results are presented in Figure HI We have two sets of plots, one where the transmitter 
can release the molecule in one of two time slots, other, where the transmitter can release the molecule 
in one of four time slots. At low velocities, the mutual information is close to log23 bits. This is because, 
at low velocities, any information encoded in the time of release of the molecule is lost. The receiver can 
however accurately estimate the number of molecules transmitted. With two molecules, the receiver can 
decode if the number of molecules transmitted was one or two or zero. However, this is because, we wait 
for infinite time at the receiver. The probability distribution function which attains the maximum mutual 
information at low velocities assigns, roughly, a probability of | to the events of releasing one or two or 
no molecules. 

At very high velocities, information encoded in both the time and number of molecules released is 
retained through the propagation. Hence, a maximum of log^IimMtH) bits can be conveyed at high 
velocities. 

In Tables IB [II] and [In] we list the mutual information maximizing input distributions for the case of 
release of two molecules in two time slots. Tables |lVl |V] and |VI] list the input distributions for the case 
of release of two molecules in four time slots. As expected, at low velocities, the total probability of 
releasing one, two or zero molecules is roughly one third each. The molecules, to minimize uncertainty, 
are transmitted 'far apart'. 

It is however surprising to note that for reasonable velocities when two molecules are released, they 
are both to be released in the same time slot. This may be explained by the fact that, due to diffusion, 
molecules can arrive out of order and the timing information is lost. Transmitting both molecules at once 
avoids this confusion. This is also an important result; if this trend is to hold true for the release of 
multiple molecules, then we could consider only those schemes wherein all the molecules are released in 
one of the time slots, and where information is encoded only in the time slot in which all the molecules 
are released. 
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TABLE I 

Mutual information maximizing input distribution when two molecules are released in one of the two possible 

slots or not released at all, v = 10~^, d = 0.05 





PiX2 = 1) 


P{X2 = 2) 


PiX2 = 0) 


P(Xi = 1) 
P{Xi = 2) 
P(Xi = 0) 


0.1424 






0.1939 



0.1412 
0.1921 
0.3303 


TABLE n 
V = 10-^ d = 0.2 




P{X2 = 1) 


P{X2 = 2) 


P{X2 = 0) 


P(Xi = 1) 
P{Xi = 2) 
P{Xi = 0) 


0.1382 






0.2113 



0.1299 
0.2035 
0.3171 


TABLE III 
v = 10, d = 0.2 OR 0.05 




P(X. = 1) 


P{X2 = 2) 


P{X2 = 0) 


P(Xi = 1) 
P(Xi = 2) 
P(Xi = 0) 


0.1667 




0.1667 
0.1667 



0.1667 
0.1667 
0.1667 



TABLE rV 

Mutual information maximizing input distribution when two molecules are released in one of the four possible 

slots or not released at all, v = 10~^, d = 0.05 





P{X2 = 1) 


P{X2 = 2) 


P{X2 = 3) 


P{X2 = 4) 


P{X2 = 0) 


P(Xi 


= 1) 


0.1395 











0.1313 


P{Xi 


= 2) 

















P{Xi 


= 3) 

















P{Xi 


= 4) 











0.2094 


0.2022 


P{Xi 


= 0) 














0.3176 



TABLE V 
V = 10"^ d = 0.2 





P{X2 = 1) 


PiX2 = 2) 


P(X2 = 3) 


P{X2 = 4) 


P(X2 = 0) 


P{Xi 


= 1) 


0.1345 











0.1256 


P(Xi 


= 2) 





0.0305 








0.0122 


P(Xi 


= 3) 








0.0129 








P(Xi 


= 4) 











0.2052 


0.1964 


P(Xi 


= 0) 














0.2827 








TABLE VI 












v = 10,d = 


0.2 OR 0.05 








P{X2 = 1) 


P{X2 = 2) 


P{X2 = 3) 


P{X2 = 4) 


P{X2 = 0) 


P(Xi 


= 1) 


0.0667 


0.0667 


0.0667 


0.0667 


0.0667 


P(Xi 


= 2) 





0.0667 


0.0667 


0.0667 


0.0667 


P{Xi 


= 3) 








0.0667 


0.0667 


0.0667 


P(Xi 


= 4) 











0.0667 


0.0667 


P(Xi 


= 0) 














0.0667 
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Fig. 9. Variation of mutual information with velocity for the case when the transmitter is allowed to release at most two molecules. The 
mutual information maximizing input distributions at the extreme points of the graph are given in Tables |l] [III [III] IIVI Ivl and I VII 



V. Relationship to achievable information rates and capacity 

When pulse-position modulation is used, symbols are normally transmitted consecutively. That is, if 
the duration of a symbol is T, then the first symbol is transmitted on the interval [0,T), the second on 
the interval [T, 2T), and so on. However, for the Brownian motions considered in Section HIl molecules 
transmitted during a given interval may arrive during a later interval, causing inter-block interference. In 
this paper, we have avoided this problem by only considering symbols transmitted in isolation, disregarding 
inter-block interference. 

In fact, for a fixed input distribution, our information results lead to an upper bound on the mutual 
information under consecutive symbol transmission. To show this, let X represent the alphabet of allowed 
symbols. For simplicity, suppose that a symbol is composed of the release of a single molecule, although 
this assumption can be relaxed without changing the argument. Then we will assume that A" is a finite, 
discrete list of allowed molecule release times on the interval [0,T); the cardinality \X\ gives the number 
of allowed release times. Further, there exists a discrete input distribution, with pmf px{x), over X. Let 
y represent the corresponding set of channel outputs, given a single symbol input to the channel, and 
disregarding inter-block interference. Since 3^ is the arrival time of a single molecule transmitted on the 
interval [0, T), then clearly y = [0, oo), and nothing changes if y is quantized. 

Let X = [xi,X2, ■ ■ ■ ,Xn] e X'^ and y = [yi,y2, ■ ■ ■ ,yn] ^ 3^" represent vectors of channel inputs and 
outputs, respectively, for n uses of the channel in isolation; throughout this section, we will assume that 
Xi is independent and identically distributed (IID) for each i. Suppose the symbols in x are transmitted 
consecutively. Then the resulting sequence of molecule release times can be written r = [ri, r2, . . .], where 

r, = Xi + {i- 1)T. (17) 

Since Xj G [0, T), clearly rj G [{i — 1)T, iT). Note that the mutual information per unit time of the channel 
is given by /(X; Y)/T, which is calculated for given X and px{x)- 
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Let Ui represent the arrival corresponding to rj. Since is a time-delayed version of Xi, and yi is the 
arrival corresponding to Xi, from (flTI) we have 

Mi = yi + {i- l)T. (18) 

The corresponding vector is u = [ui, U2, . . . , However, the receiver does not observe u directly - 
instead, it observes w, where 

w = sort(u), (19) 

and where the function sort(-) sorts the argument vector in increasing order. That is, while information 
symbol Xi corresponds to arrival time Ui, it is potentially unclear which element of x corresponds to 
arrival time Wi. 

Since the length-n vectors of consecutive input symbols r and sorted outputs w are random variables, 
we can write the mutual information between them as /(R; W). However, we are more interested in the 
mutual information per unit time I{R; W), which is given by 

I{R;W) = lim— ^/(R;W) 

n->-oo nl + 

= lim 4F^(R;W)-e, (20) 

where nT represents the total time to transmit n symbols, 5 is the extra time after nT required to wait 
for all remaining molecules to arrive, and 

^ '/(R;W). (21) 



nT nT + S 

We let 5 = log n, so that lim„_j.oo ^ = 00 (which is long enough time for all molecules to arrive with 
probability 1). With this in mind, we have the following result: 
Theorem 1: 

-I{X-Y)>I{R-W). (22) 



Proof: From (|20]) . since e is positive, 

I{R-,W)< lim _L-/(R;W). 

n-i-oo nl + 

Then we have that 

/(X;Y) = /(R;U) (23) 
> /(R;W), (24) 

where the first equality follows from (fT7l) -(fT8l). since r and u are bijective functions of x and y, 
respectively; and the second inequality follows from the data processing inequality (e.g., see [25l) and 
(fT9l ). Finally, since Xj, yi and Xj, yj are independent for any i ^ j, /(X; Y) = nI(X; Y), and the theorem 
follows. ■ 
Note that the result in Theorem [T] bounds the mutual information, and thus applies to each set X and 
input pmf px{x)', however, we can also show that the result applied to capacity. Let Cm represent the 
maximum of /(X; Y) where \X\ = m, i.e., 

Cm= max I{X;Y). (25) 

The capacity of the channel uses in isolation is then given by 

C = hm Cm. (26) 
It remains to show that this limit exists, which we do in the following result. 
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Theorem 2: C exists, and is finite, if < f , C, 7" < oo. Furthermore, if maxp^^^.) I{R] W) represents 
the capacity of I{R] W) under IID inputs, then 

C 

msixI{R]W) < -. (27) 

Px{x) T 

Proof: To prove the first part of the theorem, we proceed in two steps. 

1) Cm is a nondecreasing sequence. For each m, either: the maximizing distribution px{x) (or every 
maximizing distribution, if not unique) satisfies px{x) > for all x E X; or px{x) = for at 
least one x E X {in at least one maximizing distribution, if not unique). If the former is true, then 
Cm > Cj for all j < m; if the latter is true, then Cm = Cm-i- Thus, Cm is nondecreasing in m. 

2) Cm is upper bounded. We can write /(X; Y) = H{Y) - H(Y\X) = H{Y) - H{N), where H{N) 
is the entropy of the first arrival time. We can upper bound H{Y) independently of m as follows. 
If the pdf of y is fviv), then H{Y) = -E'[log2 l//y(y)], where E[-] is expectation. If g{y) is any 
valid pdf of y, then by a well-known property of entropy, H(Y) < -E'[log2 l/g{y)] (with equality 
when g(y) = friy))- Pick g{y) = e^^ (supported on y = [0, oo)), the exponential distribution with 
unit mean. Then H{Y) < E[y] logg e, which is finite if E[y] is finite. Finally, E[y] = E[x] + E[n] < 
T + E[n], and E[n] is known to be finite if < D, C < oo [22J. 

Since Cm is a nondecreasing, upper bounded sequence, it must have a finite limit. 

To prove the second part of the theorem, note that Theorem [T] applies to all input distributions pxix); 
thus, it also applies to the one maximizing I{R; W). As a result, since C exists and is finite (from the 
first part of the theorem), it is a nontrivial upper bound on maxp_^(2;) I{R; W). ■ 
In [fT2| . it was shown that the mutual information cannot be tractably computed in general for "sorting" 
channels, i.e., those with outputs given by (fT9l ). Since /(X; Y) can be calculated relatively easily, the 
results from Theorems [T]l2] give us useful information about the capacity of a practical system. 



VI. Conclusions and Future Work 

In this paper, a framework was constructed to study data rates that can be achieved in a molecular 
communication system. A simple model was considered for the communication system, consisting of a 
transmitter and receiver separated in space, immersed in a fluid medium. The rates achieved by a simple 
pulse-position modulation protocol were analyzed, where information is encoded in the time of release of 
the molecule. These results were extended to two molecules wherein the optimal distribution reverted to 
the PPM protocol. While preliminary, the results of this work suggest practical data transmission strategies 
depending on the value of the drift velocity. 

Given the preliminary nature of this work, there are many interesting related problems. For example, 
it would be useful to consider the limitations inherent in molecular production and detection: precise 
control over release times and amounts, and precise measurement of arrival times, may not be possible; 
more realistic communication models could be produced. Furthermore, the communication architecture 
of molecular communication systems may be considered; for instance, in order to achieve the mutual 
information results given in this paper, error-correcting codes must be used; an appropriate modulation and 
coding strategy for molecular communication needs to be identified. Finally, channel estimation techniques 
need to be derived in order to cope with unknown parameters, such as an unknown drift velocity. Much 
work remains to be done to understand molecular communication from a theoretical perspective, which 
presents many interesting and exciting challenges to communication researchers. 
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